Image reproduction apparatus having function of displaying synthesized data by varying part of original image data

ABSTRACT

An image displaying apparatus includes an image storing unit storing image data containing two or more objects. An extracting unit extracts individual data from the object of the image data. An input unit inputs additional partial image data, and attribute data representing a corresponding relationship of at least part of the image data to the additional partial image data. An associating unit associates the additional partial image data with an individual data of the image data, using the attribute data input via the input unit. A synthesizing unit modifies a data content regarding at least part of the image data, which is associated by the associating unit, and synthesizes the modified content with the image data stored in the image storing unit.

BACKGROUND OF THE INVENTION

[0001] The present invention relates to an image reproduction apparatusand an image reproducing method, as well as a recording medium in whicha program for a computer to carry out the image reproduction process, isstored.

[0002] For example, there has been proposed an image reproductionapparatus for reproducing synthesized image data as contents having datastructures of two or more different types, which are represented byobject data used in an MPEG 4 mode.

[0003]FIG. 11 is a diagrams showing a device for encoding image datacontaining conventional various data structures, and a structure of thedevice which reproduce such image data. In an image encoding device 207,various object data 201-1 to 201-5 to be handled are input, and they aresubjected to a multiplying process by a multiplying process 202, andthus image data is formed. In the conventional image reproductionapparatus 208, these image data are subjected to multiple separation bya multiple processing portion 203, to be separated into individualobject data 209-1 to 209-4, and an image to be displayed is formed onthe basis of scene description object data 209-5. In some cases, thesynthetic display at the synthetic display section 204 is controlled onthe basis of event data from a user interface 205, and the result isoutputted as an output 206.

[0004] The above-described conventional image reproduction apparatusmerely reproduces image data which was formed in advance, as it is andas it is determined. Further, even for a partial little change in data,such a newly changed data, which reflects the changed portioninstantaneously, cannot be reproduced unless the image data is reedited.Further, for example, if image data which have such a relationship as ofa background and an object is synthesized and displayed, it becomesimpossible to know which portion is the object if the user tries tooperate the object, and therefore the operation is not very much easy.

BRIEF SUMMARY OF THE INVENTION

[0005] Therefore, the object of the present invention is provide animage reproduction apparatus, an image reproduction method and recordingmedium, which can reproduce data while part of the data is updated asnew image data, without reediting the original image.

[0006] In order to achieve the above-described object, there is providedaccording to the first aspect of the present invention, an imagedisplaying apparatus comprising:

[0007] an image storing unit storing image data containing two or moreobjects;

[0008] an extracting unit extracting individual data from the object ofthe image data;

[0009] an input unit inputting additional partial image data, andattribute data representing a corresponding relationship of at leastpart of the image data to the additional partial image data;

[0010] an associating unit associating the additional partial image datawith an individual data of the image data, using the attribute datainput via the input unit; and

[0011] a synthesizing unit modifying a data content regarding at leastpart of the image data, which is associated by the associating unit, andsynthesizing the modified content with the image data stored in theimage storing unit.

[0012] Further, there is provided according to the second aspect of thepresent invention, an image reproduction method for extractingindividual data from image data containing two or more different types,and synthesizing them to be displayed; the method comprising:

[0013] an input step for inputting additional partial image data whichis different from the image data, and attribute data representing acorresponding relationship of at least part of the image data to theadditional partial image data;

[0014] an associating step for associating the additional partial imagedata with an individual data of the image data, using the attribute datainput in the input step; and

[0015] a synthesizing and reproducing step for changing a data contentregarding at least part of the image data, which is associated in theassociating unit, and synthesizing the changed content with other inputimage data, to be reproduced.

[0016] Further, there is provided according to the third aspect of thepresent invention, a recording medium storing an image reproductionprogram for extracting individual data from image data containing two ormore different types, and synthesizing them to be displayed; the imagereproduction program comprising:

[0017] an input process for inputting additional partial image datawhich is different from the image data, and attribute data representinga corresponding relationship of at least part of the image data to theadditional partial image data;

[0018] an associating process for associating the additional partialimage data with an individual data of the image data, using theattribute data input in the input step; and

[0019] a synthesizing and reproducing process for changing a contentregarding at least part of the image data, which is associated in theassociating unit, and synthesizing the changed content with other inputimage data, to be reproduced.

[0020] Additional objects and advantages of the invention will be setforth in the description which follows, and in part will be obvious fromthe description, or may be learned by practice of the invention. Theobjects and advantages of the invention may be realized and obtained bymeans of the instrumentalities and combinations particularly pointed outhereinafter.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

[0021] The accompanying drawings, which are incorporated in andconstitute a part of the specification, illustrate presently preferredembodiments of the invention, and together with the general descriptiongiven above and the detailed description of the preferred embodimentsgiven below, serve to explain the principles of the invention.

[0022]FIG. 1 is a diagram showing a basic process flow of an imagereproduction apparatus according to an embodiment of the presentinvention;

[0023]FIG. 2 is a diagram showing the structure of the imagereproduction device which executes reproduction process of theembodiment;

[0024]FIG. 3 is a flowchart illustrating the process of associating andobject data changing unit, shown in FIG. 2;

[0025]FIG. 4 is a diagram illustrating a specific example of the processin the associating and object data changing unit 309;

[0026]FIGS. 5A and 5B are diagrams illustrating that part of image datais changed by the associating and object data changing process;

[0027]FIG. 6 is a flowchart illustrating a first alternative example ofthe present embodiment;

[0028]FIG. 7 is a diagram illustrating the first alternative examplewith reference to a specific example;

[0029]FIG. 8 is a flowchart illustrating the second alternative exampleof the embodiment of the present invention;

[0030]FIG. 9 is a diagram showing an example of the list displayed inthe second alternative example;

[0031]FIG. 10 is a diagram illustrating the procedure for selectivelylisting original data on a display; and

[0032]FIG. 11 is a diagram showing the structure of the conventionalimage reproducing apparatus.

DETAILED DESCRIPTION OF THE INVENTION

[0033] First, an embodiment of the present invention will now bedescribed briefly. In this embodiment, in addition to that image data isinput to the image reproduction apparatus to reproduce it, additionalattribute data or additional partial image data are input, the attributedata or partial image data, and individual data within the image dataare associated with each other, and image data is synthesized andreproduced on the basis of the associated data. In this manner, part ofthe original data is updated without reediting the original image, andimage data can be reproduced as new image data. Further, thesynthesizing display method can be varied on the basis of the attributedata uniquely owned by the user, and therefore functions of notdisplaying an object or the like which should be hidden, displaying alist of objects, and the like can be achieved. Further, the outline ofpartial image data which should be drawn to the user's attention isdisplayed with emphasis, or the partial image is flashed at a constanttime cycle to draw the user's attention, thus providing a smooth imageoperating environment.

[0034] An embodiment of the present invention will now be described indetail with reference to drawings. FIG. 1 is a diagram illustrating abasic processing flow of an image reproduction apparatus according to anembodiment of the present invention. The present system includes aprocess 101 for inputting original image data 100 having two or moredifferent data structure, an image parser process 102 for fetchingvarious types of individual data by parsing the original image data 100,a process 103 for inputting, apart from the original image data 100,attribute data or partial image data, as additional data 106, a process104 for associating the original image data 100 with additionalattribute data or additional partial image data 106, and an imagesynthesizing and display process 105 for outputting display data 107used for synthesizing and reproducing images on the basis of theassociated results or for displaying them with emphasis.

[0035]FIG. 2 is a diagram showing the structure of an image reproductionapparatus 308 which carries out the above-described reproductionprocess. An original image data 301 inputted is subjected to multipleseparation by a multiple separation unit 303, and thus individuallyseparated object data 310-1 to 310-5 are obtained. The associating andobject data changing unit 309 associates separately input attribute dataor partial image data 302 to, and changes individual data on the basisof the association data. The synthesizing display unit 304 synthesizesthe object data 310-1 to 310-4 on the basis of the scene descriptionobject data 310-5 which instructs the way the data are synthesized anddisplayed, and outputs the synthesized data as new image data (306).Further, as in the conventional apparatus, the synthesized display canbe changed in accordance with the instruction of the user interface 305.

[0036] Here, the process at the associating and object data varying unit309 will now be described in detail with reference to flowchart shown inFIG. 3. Attribute data or partial image data inputted as additional datahas information of ID numbers respectively assigned to the objects ofthe original image data in advance, and the information indicates whenobject data should be changed. Therefore, as the information and the IDnumber of each object fetched after being subjected to the multipleseparation, are identified with each other, the object to be processedcan be found.

[0037] First. In Step S401, it is judged if unprocessed attribute dataand partial image data are present. If they are present, the operationproceeds to step S402, where the object data whose ID number matches issearched, and it is confirmed if an object to be subjected to theassociation is present. When an object whose ID number matches is found,the object data to be processed is changed using the attribute data orpartial image data in step S403. In the data changing operation, all ofthe object data may be changed or they may be partially changed. Inconnection with the changing, necessary information is reflected onscene description object data. It is possible that some of the attributedata contains the instruction of changing only the scene descriptionobject data.

[0038] The changed object is synthesized and displayed as image datawhich is different from the object data before the changing, which wasseen by the user by then. However, it is difficult for the user torecognize the renewed object immediately, and therefore it is possiblethat the renewed object obtained by changing the scene descriptionobject data, is emphasized in the display.

[0039] In order to realize the emphasis display, the object emphasizingmeans which is already realized by the conventional image reproductionapparatus may be used, or the data processing which can make the displayemphasized may be carried out on the object data itself to be replaced.

[0040]FIGS. 4 and 5 are diagrams illustrating the process in theassociating and object data changing unit 309 with reference to aspecific example.

[0041] As shown in FIG. 4, as the first image data to be input to theimage reproduction apparatus, various types of original image objects(object 101, object 102, object 103 and object 104) stored in therecording medium 10 such as CD-ROM or DVD are considered. Further, asthe second image data to be input to the image reproduction apparatus,various types of additional partial image objects (objects to bechanged) (object 111, object 112, object 113) and change instructions asadditional attribute data (replace 101→111, replace 102→112, replace101→113) are considered. In the change instruction, the numeral locatedon the left-hand side of symbol (→) indicates the object to be changed,and the numeral located on the right-hand side of symbol (→) indicatesthe object which has been changed. The second image data is stored inthe memory medium 20 such as floppy disk, CD-ROM, DVD or hard disk, andsent in a different route (for example, mail). Or the data is downloadedfrom the Internet and stored in the hardware.

[0042] The image reproduction apparatus of the embodiment reads out thefirst image data stored in the memory medium 10 and the second imagedata stored in the memory medium 20, and the two image data areassociated with each other, and the object data is changed by theassociation and object data changing unit 309 equipped in the apparatus.

[0043] More specifically, the association and object data changing unit309 associates objects 101, 102 and 103, out of inputted objects 101,102, 103 and 104, which are to be changed by the change instruction,with partial image objects 111, 112 and 113, and then changes to thepartial image object 111, 112 and 113. After the change process, thedata are synthesized and displayed on the synthesizing and display unit304.

[0044]FIG. 5A shows a state of an object before being changed, and FIG.5B shows a state of the object which has been changed by the changeprocess. As can be seen from these figures, the object 101 is changed toobject 111 and the object 104 is displayed as it is without beingchanged since there is no instruction of changing.

[0045] Next, the first alternative example of the above-describedembodiment will now be described. When there is a request of displayinginternal objects only to a particular user, it is required to have afunction of associating a user and an object to each other and carryingout verification by any means. In order to achieve this, the object isencrypted in advance into a coded number. More specifically, as shown inFIG. 6, it is judged as to whether or not there is additional attributedata or additional partial image data, which has not been processed(step S501) and whether or not there is an object to be associated with(step S502). Then, it is judged whether or not the object which has beenassociated with can be decoded with the personal identification codeowned by the user (step S503). If it can be decoded, the object isdecoded, and the object after being decoded and the scene descriptionare changed (step S504).

[0046] A specific example of the first alternative example will now bedescribed. Here, as the first image data to be input to the imagereproduction apparatus, various types of objects (object 101, object102, object 103 and object 104) and access permission instructions(Available 101 “X$$%&”, Available 104 “X$$%&”) are considered. In theabove access permission instruction, X$$%& indicates a particularpersonal security code or cryptic code.

[0047] Further, as the second image data to be input to the imagereproduction apparatus, various types of additional partial imageobjects (object 111, object 112 and object 113), change instructions(Replace 101→111, Replace 102→112 and Replace 101→113), and accesspermission instructions (Available 101 “X$$%&”, Available 104 “X$$%&”)as additional attribute data are considered.

[0048] In this alternative example, the associating and object datachanging unit 309 associates the objects 101, 102 and 103, out of theinputted objects 101, 102, 103 and 104, which should be changed underthe above change instruction, with the partial image objects 111, 112and 113, and then changes to the image objects 111, 112 and 113. Here,it is checked if there is an object of the cryptic number which matchesthe personal security code (user cryptic code) 30 input from the userinterface, with reference to the access permission instruction, and onlythose of the objects which have been judged to match are displayed.

[0049] In the above-described embodiment, when, of the second imagedata, at least change instruction and access permission instruction areread from a memory medium to the apparatus in advance, the processingtime can be shortened.

[0050] Next, the second alternative example of the embodiment will nowbe described. As to the memory medium 10 or 20, when there are variousobjects present in the medium, the user, in some cases, would like toknow what sort of objects are present. Therefore, here an object isfetched in advance in the form of a still image (or motion image), and atable list is formed to be displayed, thus improving the visibility.

[0051] More specifically, as shown in FIG. 8, while an image streamconsisting of a plurality of image objects is parsed (decoded) from thetop, it is judged whether or not there is any unprocessed object presentin the image stream (step S601) and for example, only the image objectswhich are not the background are checked, and the presence thereof isconfirmed. Further, as shown in FIG. 9, of a plurality of image objects(objects A, B, C, . . . ) which have been confirmed to be present, forexample, a still image 701 of the top frame is extracted and displayedas a list together with a frame number 702 (step S602). When the processof the image stream finishes until its end, the list is recorded (stepS603). It should be noted that in place of the frame number, the framemay be converted into time, which is displayed.

[0052]FIG. 10 is a diagram illustrating the procedure of listingoriginal image data selectively. Here, the original image data 400inputted is multiply separated into various types of object data by themultiple separation unit 401, and then particular objects are extractedon the basis of the instruction from the user 403 or the instruction ofdefault, in the object extraction unit 404. After that, the extractedobjects are synthesized, and displayed as a list (405).

[0053] It should be noted that image data displayed in a list may not beoriginal image data, but may be object data to be changed. Or these twodata may be input at the same time. When they are inputted at the sametime, the change results by the object data to be changed may bedisplayed on a list, or both image objects may be displayed on a list.Further, what is synthesized may not be a still image, but a motionstream data.

[0054] In the case where an object is encoded so that only theparticular user is allowed to access the object, it is displayed on thebasis of the information which can be accessed by the objectinformation. For example, in the case where the size of the object isonly in width and height, it is expressed by a rectangular shape,whereas in the case where the shape information can be accessed, onlythe shape is displayed.

[0055] According to the present invention, even if image data has beenedited and formed once, it can easily be partially changed andreproduced after that, and therefore the image data can be enjoyedwithout looking out of date. Therefore, it becomes possible to providehighly personalized services for individual users. Further, an objectwhich has a special significance is displayed with emphasis so that theuser can recognize it, and thus if there is a newly updated data presentwithin a screen, it can be easily visually spotted.

[0056] Additional advantages and modifications will readily occur tothose skilled in the art. Therefore, the invention in its broaderaspects is not limited to the specific details and representativeembodiments shown and described herein. Accordingly, variousmodifications may be made without departing from the spirit or scope ofthe general inventive concept as defined by the appended claims andtheir equivalents.

1. An image displaying apparatus comprising: an image storing unitstoring image data containing two or more objects; an extracting unitextracting individual data from the object of the image data; an inputunit inputting additional partial image data, and attribute datarepresenting a corresponding relationship of at least part of the imagedata to said additional partial image data; an associating unitassociating said additional partial image data with an individual dataof said image data, using the attribute data input via the input unit;and a synthesizing unit modifying a data content regarding at least partof the image data, which is associated by said associating unit, andsynthesizing the modified content with the image data stored in saidimage storing unit.
 2. An image display apparatus according to claim 1,wherein said attribute data contains a change instruction for changingat least part of said image data to said additional partial image data.3. An image display apparatus according to claim 1, wherein saidsynthesizing unit includes a synthesizing and displaying unit fordisplaying with emphasis the section of said image data, which has beenchanged to said additional partial image data.
 4. An image displayapparatus according to claim 3, wherein said synthesizing unit displayswith emphasis an outline of the section changed to said additionalpartial image data.
 5. An image display apparatus according to claim 3,wherein said synthesizing and displaying unit flashes the section whichhas been changed to said additional partial image data at constantintervals.
 6. An image display apparatus according to claim 2, whereinsaid attribute data further contains an access permission instructionfor permitting an access only to particular data out of said additionalpartial image data, and said synthesizing unit includes a synthesizingand display unit for determining whether or not the section which hasbeen changed to said additional partial image data should be displayedon the basis of said access permission instruction and a personalsecurity code input by the user.
 7. An image display apparatus accordingto claim 1, wherein said synthesizing unit includes a synthesizing anddisplaying unit for displaying in a predetermined form contents of atleast one of said image data and said additional partial image datainput via said input unit.
 8. An image display apparatus according toclaim 1, wherein said synthesizing unit includes a synthesizing anddisplaying unit for selecting data designated from outside only, fromsaid image data, and displaying contents of selected data in apredetermined form.
 9. An image display apparatus according to claim 1,wherein said synthesizing unit includes a synthesizing and displayingunit for displaying in a predetermined form contents of the section ofsaid image data, which has been changed to said additional partial imagedata.
 10. An image reproduction method for extracting individual datafrom image data containing two or more different types, and synthesizingthem to be displayed; said method comprising: an input step forinputting additional partial image data which is different from theimage data, and attribute data representing a corresponding relationshipof at least part of the image data to said additional partial imagedata; an associating step for associating said additional partial imagedata with an individual data of said image data, using the attributedata input in said input step; and a synthesizing and reproducing stepfor changing a data content regarding at least part of said image data,which is associated in said associating unit, and synthesizing thechanged content with other input image data, to be reproduced.
 11. Animage display method according to claim 10, wherein said attribute datacontains a change instruction for changing at least part of said imagedata to said additional partial image data.
 12. An image display methodaccording to claim 11, wherein said synthesizing and reproducing stepincludes a synthesizing and displaying step for displaying with emphasisthe section of said image data, which has been changed to saidadditional partial image data.
 13. An image display method according toclaim 12, wherein said synthesizing and displaying step displays withemphasis an outline of the section changed to said additional partialimage data.
 14. An image display method according to claim 12, whereinsaid synthesizing and displaying step flashes the section which has beenchanged to said additional partial image data at constant intervals. 15.An image display method according to claim 11, wherein said attributedata further contains an access permission instruction for permitting anaccess only to particular data out of said additional partial imagedata, and said synthesizing and reproducing step includes a synthesizingand display step for determining whether or not the section which hasbeen changed to said additional partial image data should be displayedon the basis of said access permission instruction and a personalsecurity code input by the user.
 16. An image display method accordingto claim 10, wherein said synthesizing and reproducing step includes asynthesizing and displaying step for displaying in a predetermined formcontents of at least one of said image data and said partial image datainput via said input unit.
 17. An image display method according toclaim 10, wherein said synthesizing and reproducing step includes asynthesizing and displaying step for selecting data designated fromoutside only, from said image data, and displaying contents of selecteddata in a predetermined form.
 18. An image display method according toclaim 10, wherein said synthesizing and reproducing step includes asynthesizing and displaying step for displaying in a predetermined formcontents of the section of said image data, which has been changed tosaid additional partial image data.
 19. A recording medium readable by acomputer, storing an image reproduction program which contains aninstruction executed by the computer, for extracting individual datafrom image data containing two or more different types, and synthesizingthem to be displayed; said image reproduction program comprising: aninput process for inputting additional partial image data which isdifferent from the image data, and attribute data representing acorresponding relationship of at least part of the image data to saidadditional partial image data; an associating process for associatingsaid additional partial image data with an individual data of said imagedata, using the attribute data input in said input step; and asynthesizing and reproducing process for changing a content regarding atleast part of said image data, which is associated in said associatingunit, and synthesizing the changed content with other input image data,to be reproduced.
 20. A recording medium according to claim 19, whereinsaid attribute data contains a change instruction for changing at leastpart of said image data to said additional partial image data.